The invention relates to data eyeglasses and to a process for determining the pose of data eyeglasses.
Nowadays, data eyeglasses (sometimes also referred to as a head-mounted display, HMD) are known, with the aid of which information can be displayed to the wearer of the data eyeglasses. In this case, the data eyeglasses are worn on the head like conventional eyeglasses, which are used as a visual aid. However, in contrast to conventional eyeglasses, the data eyeglasses include a display which, when the data eyeglasses are worn, is arranged close to the user's eye or eyes. The display may comprise two partial displays, specifically one for each eye. In the display, information can be displayed to the user in the form of text, graphic representations or mixtures thereof. In particular, the display can be partially transparent, thus may be further developed such that the wearer can also recognize the environment behind the display. Particularly preferably, the information or instructions or images are contact-analogously displayed to the wearer, which is sometimes also called “augmented reality”. In this case, the information is displayed to the wearer of the data eyeglasses in a location that is oriented according to the location of an object in the environment, thus, for example, adjoining the object or superimposed on the latter. For implementing the contact analogy, typically the position of the object in the environment and the pose of the data eyeglasses in relation to the object have to be known, thus the position of the data eyeglasses and the alignment of the data eyeglasses.
Data eyeglasses have become known which also include a camera, which takes pictures in the viewing direction of the wearer of the data eyeglasses; see, for example, International Patent Document WO 2013/012914.
Data eyeglasses can also be used in vehicles in order to display contact-analog information. This information may concern other traffic participants or objects in the vehicle. In order to be able to display contact-analog information, the pose of the data eyeglasses also has to be known here in most cases.
Inertial sensors (acceleration, gyroscope, compass) are known, which can determine a relative orientation of an electronic device, among other things, typically also of data eyeglasses. However, inertial sensors are subject to noise and to a temporal drift. They are mainly unsuitable in the vehicle because of the proper motion of the vehicle, since only the superposition of the vehicle motion and head motion is sensed. These sensors do not supply information regarding a position, but only concerning the relative orientation. An absolute referencing of the orientation requires a magnetometer, which cannot be used in the vehicle context because of strong interferences.
It is an object of the present invention to provide data eyeglasses by which the pose of the data eyeglasses in the vehicle can be determined. Likewise, it is an object of the present invention to achieve this pose determination in vehicles as accurately and technically reliably as possible.
The task of the invention is achieved by data eyeglasses in accordance with embodiments of the invention.
An aspect of the invention relates to data eyeglasses, having a first display, a camera, and electronic computing devices, wherein the data eyeglasses are configured for implementing the steps of photographing the surroundings of the data eyeglasses by use of the camera, detecting an image in the photos of the camera that is displayed by a second display; detecting that the displayed image includes a marker for the pose determination of the data eyeglasses, recognizing the marker, determining the pose of the data eyeglasses by use of the detected marker.
Furthermore, the computing unit may be configured for detecting the second display in the photos of the camera. The alignment and position of the display and thereby of the displayed marker are typically predefined, known or are provided. The determination of the pose can be carried out particularly relative to the alignment and position of the second display. The pose of the data eyeglasses can describe the alignment of the data eyeglasses in a three-dimensional space and the position of the data eyeglasses in a three-dimensional space. The marker may have a part or the full size of the image which, in turn, may fill in essentially the entire second display. The second display is typically a display of the interior of the vehicle, such as an instrument cluster or a display, which is (movably) installed in the dashboard and is used for presenting navigation and infotainment information.
A determination of the pose of the data eyeglasses is disclosed that is based on the photo or the photos of the camera of the data eyeglasses. In order to facilitate the pose determination, a predefined marker is displayed in a display, which marker is suitable for determining the pose of the data eyeglasses and therefore, in particular, therefore represents areas or lines delimited with respect to one another, in some implementations, similar or identical to a QR code. With the aid of the image of the marker in the photo (from the perspective of the camera of the data eyeglasses), knowledge concerning the further development of the marker and the position and alignment of the second display, conclusions can be drawn on the pose of the data eyeglasses. Methods for this purpose are known from the state of the art. However, the marker is displayed in the display in such a fashion that it is not apparent to a person's eye. This can be achieved particularly by using steganography and/or watermarking. However, the marker can be detected by way of the camera of the data eyeglasses (and, as required, by using image processing algorithms).
This permits a robust pose determination, because the absolute pose of the data eyeglasses is determined, which can also be achieved with sufficient precision by use of the camera technology currently available as well as what will be available in the future. Furthermore, no additional installations are required in the vehicle interior as markers for the pose determination. Compared to a system in which the pose of the data eyeglasses would be determined by cameras arranged in the vehicle and then first has to be transmitted to the data eyeglasses, in order to permit a contact-analog representation, the invention disclosed here allows a reduction of the latencies (delays for the adaptation of the contact-analog representation) after head motions. The reason is that the pose of the data eyeglasses is computed in the data eyeglasses themselves and does not first have to be transmitted to the data eyeglasses.
The second display is typically used for a different purpose, specifically for displaying text or graphic based information, such as an information image, that can be perceived by the person. In a typical application, the second display is installed in a dashboard of a vehicle, which second display, within the scope of a display-operating concept, is used for displaying to the driver of the vehicle (or a front passenger) selection menus, maps, navigation instructions or, for example, illustrating graphics or images. As an alternative, the display may also be a so-called instrument cluster, which typically indicates the speed of the vehicle. The presentations provided within the scope of this display-operating concept are not very suitable for photographing and determining the pose of data eyeglasses. The typical presentations are changeable and frequently comprise no clearly delimited geometric shapes. According to the data eyeglasses disclosed here and the described method, a marker is worked into the information image by means of steganography or watermarking and is them embedded. Depending on the information image, a correspondingly fitting embedding of the marker in the information image will be determined. As a result of the embedding, typically the colors and/or the brightness of areas of the information image are changed. A person cannot perceive the embedded marker in the information image; only the information image is visible to the naked eye at typical observation intervals. The marker is predefined and is provided to the data eyeglasses or is stored so that it can be retrieved by the latter. For detecting the marker in the photo of the image, as required, first a process must be used that corresponds to the steganography or water marking process used during the embedding.
In a typical implementation, the pose, thus, the position and alignment, of the second display, is taken into account for determining the pose of the data eyeglasses. The pose of the second display is usually unchangeable, and a one-time or infrequent transmission to the data eyeglasses may therefore be sufficient.
In some implementations, the data eyeglasses are further configured for receiving a notification that the marker is displayed on the second display. The data eyeglasses may also be configured for attempting the detection of the marker only in the cases in which the notification is received or for a defined time period thereafter. This permits the saving of computing capacity and energy, which increases the battery life of the data eyeglasses.
Finally, the data eyeglasses may also be configured for the contact-analog displaying of a representation on the first display as a function of the defined pose of the data eyeglasses. For this purpose, the data eyeglasses receive a presentation to be displayed and the location in the surroundings (or a reference location relative to the position of the second display or of the vehicle), in which the presentation for the user of the data eyeglasses is to appear. Corresponding to the pose determined by the eyeglasses themselves, the presentation is displayed by the data eyeglasses in correspondingly determined positions of the first display (thus, in the display of the data eyeglasses).
Another aspect relates to a vehicle including a display control and a display, wherein the vehicle is configured for implementing the steps of determining an information image to be displayed in the display; changing the information image such that a marker is embedded therein that is not visible to an average unaided human eye, wherein the marker is designed such that it permits the pose determination of data eyeglasses by use of a camera photo of the data eyeglasses; and displaying the changed information image. The changed information image is therefore the image that is displayed in the information display or the instrument cluster.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.
Identical reference symbols relate to corresponding elements in all figures.